arxiv e-print
Adaptable Agent Populations via a Generative Model of Policies
In the natural world, life has found innumerable ways to survive and often thrive. Between and even within species, each individual is in some manner unique, and this diversity lends adaptability and robustness to life. In this work, we aim to learn a space of diverse and high-reward policies in a given environment. To this end, we introduce a generative model of policies for reinforcement learning, which maps a low-dimensional latent space to an agent policy space. Our method enables learning an entire population of agent policies, without requiring the use of separate policy parameters. Just as real world populations can adapt and evolve via natural selection, our method is able to adapt to changes in our environment solely by selecting for policies in latent space. We test our generative model's capabilities in a variety of environments, including an open-ended grid-world and a two-player soccer environment. Code, visualizations, and additional experiments can be found at https://kennyderek.github.io/adap/.
Even Faster SVD Decomposition Yet Without Agonizing Pain
Neural Information Processing Systems http://nips.cc/
Professor Forcing: A New Algorithm for Training Recurrent Networks
Alex M. Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C. Courville, Yoshua Bengio
The Teacher Forcing algorithm trains recurrent networks by supplying observed sequence values as inputs during training and using the network's own one-stepahead predictions to do multi-step sampling. We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps. We apply Professor Forcing to language modeling, vocal synthesis on raw waveforms, handwriting generation, and image generation. Empirically we find that Professor Forcing acts as a regularizer, improving test likelihood on character level Penn Treebank and sequential MNIST. We also find that the model qualitatively improves samples, especially when sampling for a large number of time steps. This is supported by human evaluation of sample quality. Trade-offs between Professor Forcing and Scheduled Sampling are discussed. We produce T-SNEs showing that Professor Forcing successfully makes the dynamics of the network during training and sampling more similar.
How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD
Neural Information Processing Systems http://nips.cc/
A Benchmark for Interpretability Methods in Deep Neural Networks
Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim
Neural Information Processing Systems http://nips.cc/
Using Embeddings to Correct for Unobserved Confounding in Networks
Victor Veitch, Yixin Wang, David Blei
Neural Information Processing Systems http://nips.cc/
Contrastive Retrospection: honing in on critical steps for rapid learning and generalization in RL Chen Sun
In real life, success is often contingent upon multiple critical steps that are distant in time from each other and from the final reward.
Learning from brains how to regularize machines
Zhe Li, Wieland Brendel, Edgar Walker, Erick Cobos, Taliah Muhammad, Jacob Reimer, Matthias Bethge, Fabian Sinz, Zachary Pitkow, Andreas Tolias
Neural Information Processing Systems http://nips.cc/
Uncertainty-Aware Attention for Reliable Interpretation and Prediction
Jay Heo, Hae Beom Lee, Saehoon Kim, Juho Lee, Kwang Joon Kim, Eunho Yang, Sung Ju Hwang
Neural Information Processing Systems http://nips.cc/
Communication trade-offs for Local-SGD with large step size
Aymeric Dieuleveut, Kumar Kshitij Patel
Neural Information Processing Systems http://nips.cc/